The present techniques relate to detecting and describing lesions. More specifically, the techniques relate to automatically detecting and semantically describing lesions using a trained convolutional neural network.